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ABSTRACT 

Continued agricultural productivity hinges on understanding how to manage soil resources. A 2-week undergraduate 
introductory-level module: A Growing Concern: Sustaining Soil Resources Through Local Decision Making was collaboratively 
developed through the InTeGrate Project. InTeGrate modules and courses engage students in grand challenges of 
sustainability (e.g., agriculture, water, and climate) using active learning strategies. In A Growing Concern, students examine 
physical and chemical distinctions between intensively managed agricultural landscapes (e.g., conventional tillage) and 
natural vegetative types. They analyze geospatial and soil profile data to identify how intensive land management threatens 
soil sustainability. After exploring land practice and climate impacts on soil, they create extension-style fact sheets that provide 
recommended practices to reduce soil erosion. To maximize accessibility, we piloted the module in three settings: (1) an 
interdisciplinary Ecological Agriculture course at a Land Grant Institution, (2) a Geology of the Critical Zone course at a 4- y 
college, and (3) an Introduction to Environmental Science course at a 2- y community college. Classroom observations using 
the reformed teaching observation protocol revealed that the instructors used reformed teaching practices. Students also 
commented favorably on the hands-on nature of the module within focus group sessions. All students passed the culminating 
fact sheet, which was aligned with the rubric used in module development. Students had some difficulty interpreting site- 
specific geologic data and applying systems thinking. Revisions to instructional materials emphasized local data and greater 
systems diagramming. © 2016 National Association of Geoscience Teachers. [DOI: 10.5408/15-106.1] 
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INTRODUCTION 

More than one-third of Earth's land is covered with 
crops that support food, fiber, and fuel needs (Hooke et al., 
2012). Globally, agricultural intensification has generated 
erosion rates that exceed geologic rates of erosion by more 
than eight orders of magnitude (Hooke, 2000). Not only is 
soil destroyed faster than it is created (Montgomery, 2007), 
but the physical and chemical properties of the soil have 
been altered dramatically by conventional agricultural 
practices (e.g., Linn and Doran, 1984; Bayer et al., 2006). 
Intensification with attention to soil conservation may be the 
only long-term solution, with limited land area available for 
agricultural expansion (Ramankutty and Rhemtulla, 2012). 
Climate change exacerbates soil erosion via increased 
flooding or loss of biomass in response to drought (Nearing 
et al., 2004). We must develop adaptation strategies for 
sustainable agriculture, especially at local scales (Rosenzweig 
et al., 2007). Addressing soil sustainability into the future 
calls for educating students about the "skin of the earth" that 
supports us (Montgomery, 2007). 

To educate and inspire the next generation to address 
issues of agricultural sustainability, agricultural curriculum is 
needed outside of traditional agricultural programs (NRC, 
2009). A recent survey of programs in the United States 
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reveals that this trend is well under way and that more total 
geology, geography, and environmental science programs 
include soils content than there are total soil science 
programs (Brevik, 2009). Robust undergraduate soils curric¬ 
ulum should incorporate central habits of soils professionals, 
including fieldwork, communication, jargon, systems think¬ 
ing, authentic problems, and making connections by 
revisiting concepts in new situations (Field et al., 2011). To 
achieve this, we developed A Growing Concern: Sustaining 
Soil Resources Through Local Decision Making through the 
InTeGrate Project (Fortner et al., 2015). This module 
includes six 50-min units for independent or sequential 
implementation. Only the culminating unit requires that 
students synthesize knowledge from previous units to 
generate an agricultural extension-style fact sheet. Our goal 
is to describe the pilot of A Growing Concern and evidence for 
its impact on student learning and engagement. This 
features details about the alignment of the fact sheet 
materials and associated student work with InTeGrate 
guiding principles. We also discuss key revisions to the 
module that improved alignment with guiding principles 
and offer lessons learned for adapting or using A Growing 
Concern. 


MODULE DESCRIPTION 

A Growing Concern and other modules and courses 
developed through the InTeGrate Project provide students 
with opportunities to explore the nexus between geosciences 
and sustainability (O'Connell et al., 2016). Modules were 
designed by authors from three institutional settings, with 
the goal of expanding accessibility to students (Egger et al., 
2013); the authors of this paper developed A Growing 
Concern. Modules are aligned with measurable learning 


1089-9995/2016/64(4)/259/11 


259 


© Nat. Assoc. Geosci. Teachers 





260 Fortner et al. 


J. Geosci. Educ. 64, 259-269 (2016) 


goals. Specifically, the A Growing Concern module supports 
mastery of two learning goals: 

• Students will use geologic data to develop a plan for 
sustainable soil management in one or more agricul¬ 
tural settings. 

• Students will predict, using systems thinking, agri¬ 
cultural challenges that might result from climate 
change. 

Guiding principles for all InTeGrate modules and 
courses include addressing a geoscience-related grand 
challenge to sustainability (e.g., soil erosion and climate 
change), engaging students in interdisciplinary thinking, 
incorporating the methods and habits of geoscientists, 
analysis of credible geoscience data, and incorporating 
systems thinking. These elements are detailed fully in the 
materials development and refinement rubric available 
online (InTeGrate Project, 2015). Topics, unit learning goals, 
and activities during the pilot are summarized in Table I; the 
entire module, including all instructor and student resources 
needed for implementation, is freely available online (Fort¬ 
ner et al., 2015). Throughout the module pilot, students 
explored the differences between intensively managed 
agricultural landscapes (e.g., grazelands and conventional 
tillage) and forested or natural vegetative types. Unit 1, for 
example, highlighted landscape differences between phys¬ 
iographic regions in agricultural and nonagricultural set¬ 
tings. Students examined landscape photos and records and 
reflected on observations. Unit 2 engaged students in the 
direct exploration of soil properties, including porosity and 
permeability. Experimental results informed their ideas for 
behavior in agricultural settings. Units 3 and 4 helped 
students develop an understanding of the spatial distribution 
of soil erosion associated with land use and soil properties 
with depth. Students worked with spatial figures and soil 
profile data, respectively, and considered implications for soil 
sustainability. Unit 5 engaged students in a jigsaw activity 
exploring factors that drive erosion, including climate 
change. Mixed pedagogical approaches provided students 
with opportunities to explore soil and implications for 
agricultural sustainability from distinct angles. During the 
pilot, units were completed sequentially. But they may also 
be completed independently to achieve unit-level learning 
outcomes. 

Student mastery of all units was measured through a 
summative assessment project in Unit 6. In this culminating 
unit, students were situated in the role of agricultural 
manager. After viewing examples of effective and ineffective 
fact sheets from multiple environmental agencies, students 
synthesized lessons learned from previous units to produce 
an agricultural fact sheet. Fact sheets required the following: 
an argument for the significance of soil sustainability, a 
description of links between land management practices and 
erosivity, the predicted impacts of climate change on erosion 
rates, recommendations to minimize erosion, a call to action, 
and a description of the feasibility of recommendations for 
farmers. Format expectations included a 2-page limit, a 
balance of text and whitespace, an awareness of audience, 
and attention to grammar. These content and format 
expectations appear in the rubric handed out with the 
assignment and include exemplary, basic, and nonperfor¬ 
mance categories. The fact sheet rubric used during the pilot 


was similar to the final version in the published module 
(Fortner et al., 2015), with minor modifications described in 
the discussion. 


PILOT SETTING 

The pilot of A Growing Concern occurred at each of our 
respective institutions. Our classes were small (<16 stu¬ 
dents), with a total of 40 students in all three courses. The 
InTeGrate Project collected demographic information 
through an external server not associated with the pilot 
schools (Egger et al., 2013; Kastens et al., 2014; McConnell et 
al., 2014). In total, 35 students responded to the demo¬ 
graphic survey, but responding students did not always 
answer all questions. Survey responses have been reported 
in aggregate to respect confidentiality concerns associated 
with small numbers at each institution. Of 35 students, 11 
were taking a required course, and 21 reporting students 
were in their third or fourth year of college. Students were 
mostly female (n = 22), with eight reporting males. 

At Santa Rosa Junior College, a 2-y college, Murphy 
piloted the module in Introduction to Environmental Science 
(ENVS12), a general education science course. The course 
introduced environmental issues from a scientific perspec¬ 
tive, focusing on physical, chemical, and biologic processes 
within the Earth system; the interaction between humans 
and these processes; and the role of science in finding 
sustainable solutions. The course served primarily nonsci¬ 
ence majors, and labs were not a required component. Soils 
and climate change were part of the curricula for this course, 
and the module replaced direct instruction on these topics in 
the lecture portion of the course. 

At Wittenberg University, a 4-y liberal arts program, 
Fortner piloted the module in an introductory course. 
Geology of the Critical Zone (Geol 170). The students in 
this course were all declared nonscience majors completing 
general education requirements with the exception of one 
undeclared student who subsequently declared a geology 
major. The Geology of the Critical Zone course seeks to have 
students understand how changes in anthropogenic, tec¬ 
tonic, and climate conditions force critical zone responses 
observed in soil, water, and biogeochemical response (e.g., 
see ideas from Brantley et al., 2007). The module was linked 
to the impact of two of these forcings (anthropogenic and 
climate) on agricultural sustainability and the role of tectonic 
forcing in soil production and erosion rates. 

At Virginia Tech, a land grant research institution, 
Scherer piloted the module in the intermediate-level 
Ecological Agriculture course, which was required for an 
interdisciplinary minor in Civic Agriculture and Food 
Systems within a college of agriculture and life sciences. 
The course examined the ecologic foundations of sustainable 
agriculture practice and surveyed the principles of ecology 
and evolutionary biology in the context of civic agriculture 
and food systems. It also included an overview of both 
historic and modern sustainable agriculture practices. 
Students in this course typically do not have strong science 
backgrounds, and this module provided them the opportu¬ 
nity to work with real-world data and learn concepts that 
provide support for sustainable agriculture practices. It also 
incorporated geoscience concepts into a course that cur¬ 
rently focuses on biology content. 
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METHODS 

Our pilot was part of the larger InTeGrate project. The 
broad scope of the entire project evaluation is not described 
here. Rather, this study details the evaluation of A Growing 
Concern as it relates to understanding student outcomes 
associated with this introductory-level module. A detailed 
explanation for each method used to evaluate the impact of 
the module on students follows. 

Review of the A Growing Concern Summative 
Assessment and Student Work 

Two members of the InTeGrate Project assessment team 
(subsequently referred to as the materials reviewers), who 
were not involved in module development, examined the 
module summative assessment (i.e., the fact sheet assign¬ 
ment) and student work completed during the pilot. Both 
the assessment and the student work were scored for 
alignment with A Growing Concern module learning goals 
and InTeGrate guiding principles (i.e., addresses a grand 
challenge, addresses interdisciplinary problem, incorporates 
methods and habits of geoscientists, includes analyses of 
authentic data, and incorporates systems thinking). Each 
element (two module learning goals and five InTeGrate 
guiding principles) was rated on a rubric with a scale of 0-3. 
A score of 3 meant the element was explicitly and/or 
pervasively addressed by students, a 2 meant it was 
addressed in most of the materials, a 1 meant it was 
somewhat addressed, and a 0 meant the element was not 
addressed. Using this rubric, the materials reviewers 
separately rated the strength of the summative assessment 
assignment (Unit 6 materials provided to students) and 
actual student work. All fact sheets completed during the 
pilot were reviewed; these included individual or small-team 
submissions, depending on instructor preference. In our 
Results section, scores from reviewers are presented as 
averages as they were reported to the module authors and 
their feedback is summarized. In addition to reporting 
reviewer scores and feedback, we include examples from 
student work that illustrate the concepts discussed in the 
feedback and the percentage of student responses for which 
this concept was true. To determine these percentages, we 
systematically coded each fact sheet for evidence of this 
concept (i.e., present or absent). 

Student Performance on Fact Sheet Assignment 

In addition to the review of the fact sheets for alignment 
with the overall module and InTeGrate goals described 
earlier, student submissions of the fact sheet (n = 31) were 
graded. These summative assessment grades were deter¬ 
mined by the individual instructors using the grading rubric 
that is included in Unit 6 of A Growing Concern. This rubric 
assesses student mastery of specific content and formatting 
expectations for the fact sheet as explained in the assign¬ 
ment. The grade range for these summative assessments 
across the three pilot settings and medians for each pilot 
setting are reported. 

Module Author Reflections on Pilot 

Module authors submitted a required postpilot reflection 
following teaching A Growing Concern in their class. We 
individually reflected on what went well, considered what 
was challenging, and determined strengths and weaknesses 
of the materials from our perspective as instructors. After 
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submission of our reflections, we compared individual 
postpilot reflections to determine areas of agreement. We 
focus our description here on these shared impressions that 
contributed to decisions about final revisions of the module. 

Classroom Observations and Student Feedback 

During the A Growing Concern pilot, members of the 
InTeGrate Project who were not involved in module 
development observed two of our courses. During these 
multiple-day site visits, classroom observations of Units 2, 
3, 5, and 6 were conducted using an InTeGrate observa¬ 
tion protocol and the reformed teaching observation 
protocol (RTOP) (McConnell et al., 2014), with the goal 
of providing feedback to the module authors. The RTOP 
measures evidence for reformed teaching, as indicated by 
student construction of knowledge in class (Sawada et al., 
2002). The RTOP is a tool used to evaluate reformed 
classroom dynamics using a rubric (0, never occurred, to 4, 
very descriptive) to collect data on five subscales: lesson 
design and implementation, propositional knowledge 
(what the teacher knows and presents), procedural 
knowledge (what the students do), student-student 
interaction, and student-faculty interaction (Budd et al., 
2013). This tool has helped establish the relation between 
reformed teaching practices and student learning (Falcon¬ 
er et al., 2001; Budd et al., 2010). RTOP analysis yields a 
number between 0 and 100, with three categories of 
instruction: teacher centered (scores < 30), transitional, 
and student centered (scores > 50) (Budd et al., 2013). 
RTOP scores (when collected) and written observer 
feedback were reported to the module authors. 

Observers also conducted focus groups, which allow for 
solicitation of opinions in a social setting in which 
participants can hear and respond to the ideas expressed 
by others (Patton, 2002). Members of the InTeGrate Project 
conducted focus groups with students who participated in 
the pilot of the module in two courses. They lasted 
approximately 30 min and involved around five students 
each. One focus group was conducted following the pilot of 
Units 2 and 3, and the second followed the pilot of Units 5 
and 6. Questions solicited student opinions of the strengths 
and weaknesses of the units as they experienced them in the 
classroom. Feedback was provided to the module developers 
in narrative form with no identifying information to protect 
the confidentiality of individual students. The limitations of 
both the RTOP and the focus group interviews in this study 
are their short temporal duration (i.e., a single class). 

Assessment of Interdisciplinary Problem Solving 

During the last week of class, students from two pilot 
classes (n = 22) responded to an interdisciplinary thinking 
essay prompt: "Knowledge of Earth system interactions can 
influence how people make decisions about global chal¬ 
lenges. Identify and describe a global challenge that society 
will likely face in the next 50 years. Explain how the science 
related to that challenge informs economic, social, and/or 
political decision making related to the global challenge you 
described." Student responses were coded for the challenges 
described (e.g., water, soil, and climate) by members of the 
InTeGrate Project for the entire InTeGrate cohort (e.g., this 
module and other modules developed in the same time 
frame) {n = 180). We examined student responses in our 
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TABLE I: Overview of units piloted in A Growing Concern, including learning goals and key activities for each of the six units 
(modified from Fortner et al., 2015). 


Unit 

Learning Goals 

Key Activities 

Unit 1: Impacts of Land Use 

1. Use objective language to discuss and record 
the physical features of various landscapes 
presented in photographs. 

2. Use their observations to compare and 
contrast agricultural with nonagricultural 
landscapes. 

3. Discuss how components of Earth's systems 
(e.g., slope, rainfall, climate, soil type) can 
impact agricultural landscapes and soil 
sustainability. 

• Identified physiographic regions from an image 
of the United States from space. 

• Made independent observations of both 
agricultural and natural landscapes in 
photographs, discussed these observations, and 
then interpreted impacts of agriculture on 
landscape. 

• Diagramed relevant systems interaction- 
associated images. 

Unit 2: Soil Characteristics and 
Their Relationship to Land Use 
Practices 

1. Describe the soil properties of porosity and 
permeability. 

2. Characterize the porosity and permeability of 
a soil sample. 

3. Interpret and assess the effects of land use 
practices on the porosity, permeability, and 
erosivity of the soil. 

4. Make recommendations for sustainable 
agricultural practices in a hypothetical scenario. 

• Completed a hands-on activity exploring soil 
properties of porosity and permeability of soil 
samples representative of both agricultural and 
natural environments. 

• Used experimental results to compare and 
contrast soils from varying environments while 
modeling the role of a soil assessment expert. 

Unit 3: Exploring Natural and 
Agricultural Erosion Rates 

1. Interpret data from geospatial figures, and 
analyze erosion rates. 

2. Discuss the influence of agricultural erosion 
on soil sustainability. 

3. Confront preconceived ideas, reframe these 
ideas given new data, and reflect on that 
process. 

• Examined and discussed perceptions of erosion 
from images of mountain and agricultural 
landscapes. 

• Used geospatial figures to compare erosion 
rates associated with natural and agricultural 
landscapes in the United States (involves 
converting units). 

• Reflected on the negative implications of 
agricultural erosion on soil sustainability. 

Unit 4: Using SoilWeb to 
Investigate the Soil Beneath You 

1. Use the SoilWeb smartphone app or website 
to identify the most common soil in their 
area and sketch its soil profile (California Soil 
Resource Lab, 2015). 

2. Explain how the physical and chemical 
properties of soil relate to soil horizons. 

3. Identify which soil horizon is most important 
to fertility and how that fertility is affected by 
erosion. 

• Retrieved soil information for the soil beneath 
the students using SoilWeb. 

• Using retrieved information, drew soil horizons 
and percent soil organic matter to scale, and 
discussed implications for the depth of soil 
fertility. 

• Completed a jigsaw activity comparing local 
soil erosion rates, soil horizons, and soil 
organic matter to other sites. 

• Discussed how the speed of implementing soil 
solutions impacts society and the economy. 

Unit 5: Predicting the Effects of 
Climate Change on Soil Loss 

1. Explain how rainfall and runoff erosivity, soil 
properties, landscape characteristics, and 
agricultural practices contribute to soil 
erosion. 

2. Differentiate between natural and human 
influences on soil sustainability. 

3. Analyze, using systems thinking, how 
changes in precipitation predicted in climate 
change models for their region will impact 
erosion rates. 

• Completed a jigsaw activity exploring factors 
that drive erosion (i.e., revised universal soil 
loss equation variables). 

• Participated in a kinesthetic activity about 
influences on soil sustainability. 

• Watched a short video on climate change and 
agriculture. 

• Engaged in a guided lecture introducing 
systems diagrams and impacts of climate 
change on soil sustainability in their local 
region. 

Unit 6: Creating an Agricultural 
Fact Sheet 

1. Synthesize information about soil erosion, 
climate, management practices, and 
sustainability. 

2. Make recommendations for agricultural 
practices that can minimize erosion and 
address feasibility of these actions from the 
perspectives of key stakeholders. 

3. Use evidence to show what actions are 
needed to mitigate soil erosion, thereby 
increasing sustainability of soils. 

4. Apply the characteristics of good science 
communication to the public by creating 
user-friendly fact sheets. 

• Reviewed criteria for the summative fact sheet 
assignment. 

• Recalled what was learned in Units 1-5 that 
aligns with criteria for the summative fact sheet 
in a brain dump activity. 

• Shared brain dump with small groups. 

• Discussed both good and bad fact sheets 
examples and reflected on audience. 

• Crafted audience-friendly fact sheets that 
synthesize evidence about soil sustainability, 
climate change, and relevant management 
solutions. 
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TABLE II: Average scores of the alignment of the summative fact sheet assignment and student work with module learning goals 
and InTeGrate guiding principles. Possible scores range from 0 to 3; full alignment is equal to a 3. Scores of 2 and above reflect 
substantial alignment. Two external evaluators conducted scoring. 



Fact Sheet 
Assignment 

Student 

Work 

Module Learning Goals 

Degree to which assessment reflects ability to use geologic data to develop a plan for sustainable soil 
management in one or more agricultural settings 

2.5 

2 

Degree to which assessment reflects ability to predict agricultural challenges that might result from 
climate change using systems thinking 

2.5 

2 

InTeGrate Guiding Principles 

Degree to which assessment reflects competence explaining one or more geoscience-related grand 
challenges facing society 

3 

3 

Degree to which assessment reflects ability to address interdisciplinary problems 

2.5 

2.5 

Degree to which assessment reflects the nature and methods of geoscience and developing geoscientific 
habits of mind 

2.5 

2 

Degree to which assessment reflects use of authentic and credible geoscience data to learn central 
concepts in the context of geoscience methods of inquiry 

3 

2 

Degree to which assessment reflects ability to incorporate systems thinking 

2.5 

1.5 


pilot courses for further evidence of alignment with topics 
considered within the module. 


RESULTS 

Review of the A Growing Concern Summative 
Assessment and Student Work 

The results of the InTeGrate assessment team members' 
review of the module summative assessment (fact sheet) 
assignment and associated student work for alignment with 
module and InTeGrate goals are presented in Table II. Fact 
sheet assignment materials (what the students were asked to 
do) received scores of 2.5 or higher out of 3 on all 
components of the rubric, suggesting that the materials 
were highly aligned with the InTeGrate materials develop¬ 
ment and refinement rubric and module learning goals. 
When describing how well the fact sheet addresses module 
learning goal 1 (i.e., using geologic data to develop a 
management plan), reviewers noted "the 'fact sheet' 
provides opportunities for students to 'use geologic data' if 
students focus on interpreting data for their region." 
Similarly, in their feedback describing the alignment of the 
final fact sheet with Learning Goal 2, they stated that overall, 
"this was a great assignment because it asks students to 
predict what will happen in their region by using regional 
climate data, climate change predictions, data on soils," etc. 

Materials reviewers determined that fact sheet materials 
mostly incorporated InTeGrate guiding principles. Both 
reviewers described fact sheet prompts as explicitly refer¬ 
encing grand challenges of climate and soil sustainability 
and making interdisciplinary connections. Each of these 
criteria received a 3 in the materials review. However, both 
reviewers felt that fact sheet materials could better empha¬ 
size the nature and methods of science. This category 
received a 2, and reviewers described the fact sheet as 
focusing more on analyses than other methodology, stating 
that the materials "do emphasize that students should be 
interpreting data for the region in which they live" but that 
other connections among analyses could be made more 


explicit. They reiterated this point in their evaluation of the 
InTeGrate Guiding Principle of authentic data. Finally, the 
fact sheet assignment received a 2.5 for addressing systems 
thinking, because it largely addressed this criterion by asking 
students to specifically address three or more variables 
affecting soil sustainability. 

Materials reviewers scored student performance on fact 
sheets related to the module learning goals and InTeGrate 
guiding principles between 1.5 and 3. Student attainment of 
Module Learning Goals 1 and 2 received scores of 2. 
Learning Goal 1 emphasized the use of data, which students 
needed to incorporate to describe local soils. Materials 
reviewers noted that students often used generic geologic 
data, rather than site-specific data important to management 
decisions. To illustrate, one submission described a generic 
soil profile as follows: "The most fertile layers of soil are the 
top layers, O Horizon and A Horizon. Once these layers are 
eroded, harder, less fertile, clay-like soils then become the 
immediate topsoil." Another said, "Soil removed by erosion 
contains about 3 times more nutrients and 1.5 to 5 times 
more organic matter than the soil that remains behind." 
Only three fact sheets (10%) incorporated local soil data 
from the SoilWeb exercise. For learning goal 2 (i.e., using 
systems thinking to predict challenges to soil erosion 
resulting from climate change), materials reviewers suggest¬ 
ed that students tended to cut and paste information on soil 
and climate change from multiple sources that may or may 
not be locally relevant. For example, multiple fact sheets 
included images of soil erosion that were not tailored to the 
location their fact sheet considered. Three examples included 
maps with climate or soil information of large areas that 
were described generally, rather than highlighting a region 
of interest. Another included a graph showing change in 
global average temperatures through time, which was 
beyond the scope of a regional evaluation. Despite these 
limitations, all fact sheets contained at least some place- 
specific information, including images of crops grown in the 
region. This is important because module learning goals 


















264 Fortner et at. 


J. Geosci. Educ. 64, 259-269 (2016) 


implicitly consider place as a framework for evaluating soil 
data and climate trends for management decisions. 

Materials reviewers provided detailed feedback discuss¬ 
ing alignment of student materials with overarching 
InTeGrate guiding principles. Reviewers commented that 
all students addressed one or more grand challenges (e.g., 
soil, climate, and water) in their fact sheet assignment. Of 
these grand challenges, they were most able to discuss the 
limitations of natural resources (soil). For example, one fact 
sheet summarized, “The quality of our soil is degrading at an 
alarming rate because of the use of the role humans play. 
The use and abuse of our croplands has left our future to be 
undesirable." It went on to identify additional causes of soil 
loss associated with human activity. Another stated that "soil 
sustainability is much more important than people think and 
it has an effect on a larger group of people than most think. 
Soil erosion is a very real event that is affecting more and 
more landowners and if the proper precautions aren't taken 
to slow it down it can be very detrimental." 

Materials reviewers also assigned student work an 
average score of 2.5 on the InTeGrate principle of addressing 
interdisciplinary problems. Among fact sheet assignment 
submissions, 90% discussed at least two of the three spheres 
of sustainability. Comments from the reviewer who assigned 
a 3 noted that the best examples were able to make links 
between scientific, economic, and political issues. The 
reviewer who assigned a 2 said that although the fact sheets 
contained interdisciplinary topics, many students didn't truly 
consider the feasibility of their recommendations from the 
stakeholder perspective. When discussing feasibility, one 
example stated, "A deeper rooting system will also be 
important to combating climate change.... If plants are in 
place with deep root systems to help soak up the moisture, 
less rain wouldn't be a problem. The lessened rate of rain 
will also help the erosion rate to decrease. When these deep 
rooting plants are harvested, their roots will help to break up 
the soil and make tilling for the next growing system easier. 
This would help the soil sustainability or, the maintenance of 
soil productivity for future generations." This example and 
several other examples don't reflect on what management 
strategies might mean for the farmers using them in terms of 
cost or barriers to implement. 

Materials reviewers scored the student attainment of an 
understanding of the nature and methods of geoscience as a 
2 (mostly addressed). They noted that students explicitly 
made comparisons between agricultural practices that 
reflected an integrated understanding of soil decision 
making. All fact sheets included descriptions of the causes 
of soil erosion and the importance of sustaining soil. 
Students described key practices needed to sustain soil, 
e.g., "Crop cover can include numerous different practices. 
One way to use crop cover to eliminate the erosion is by 
planting seasonal crops, like soybeans, earlier in the spring 
so when the rain season begins there is less erosion (Nearing 
et al., 2004)." However, a reviewer wrote, "student work 
focuses largely on spatial and temporal comparisons, but less 
so on other methods," reiterating an earlier point about how 
scale was central to meeting our learning goals, and hence, a 
key habit. 

Both materials reviewers noted that materials clearly 
called for the analysis of authentic and credible geoscience 
data to learn central concepts in the context of inquiry. But 
they rated student work as a 2, saying "students were given 


latitude" in terms of the data they brought into their fact 
sheet. For example, only 40% of fact sheets included climate 
change forecasts specific to the region the fact sheet was 
targeting. However, two-thirds of the fact sheets described 
solutions that were specific to the region, and all examples 
correctly identified soil erosion responses associated with 
specific climate conditions. 

The lowest student work score, 1.5, was for the 
incorporation of systems thinking skills. One reviewer 
reported that this criterion was somewhat addressed, and 
the other determined that it was mostly addressed. One 
wrote, "Half of the submissions include 3 or more causally 
linked variables that explain soil erosion." For example, 
one fact sheet stated: "rainfall in many highly productive 
areas like the Midwestern US is predicted to decrease due 
to climate change. Although less rainfall means less water 
washing away soil, less moisture compromises soil 
integrity and decreases crop yields, decreasing the ability 
to absorb water [and] increasing erosion rates." Another 
said, "Less rain and hotter temperatures generally means 
there will be a drought. Dry soils allow soil to be held 
together less tightly. In this case, a major factor will be 
wind erosion." Yet neither the fact sheet assignment nor 
the student work included systems language like flux, 
reservoir, or feedback. 

Student Performance on Fact Sheet Assignment 

Summative assessment grades for fact sheets submitted 
in all three pilots (n = 31) ranged from 60% to 98% of the 
points available. Two pilot sites had the students complete 
the fact sheet assignment individually; median scores at 
these sites were 81% and 80%. The third site had students 
complete the fact sheet in groups of three or four, because 
this was consistent with other major assignments in the 
course; the median score at this site was 90%. 

Module Author Reflections on Pilot 

After completing our module pilot, we collectively 
identified strengths and weaknesses from our individual 
postpilot reflections (Table III). Throughout the module, we 
observed evidence of students building from foundational 
practices, like observation, into hands-on exploration of soil 
and authentic data. Overall, we felt that students successfully 
worked through the module and were able to address soil 
science within the context of agricultural sustainability. We 
agreed that students performed well on the summative fact 
sheet assignment and felt that their work aligned with our 
module learning goals and InTeGrate guiding principles. We 
also felt that students completed most activities successfully 
and made connections between units. On a unit-by-unit 
basis, we were happy that central learning goals and all 
listed strengths that tie back to those goals were met. 

Some weaknesses stood out to us. We all noticed the 
difference between the workflow of the module and our 
normal teaching style. We all chose either prework or 
homework in our normal workflow, and moving between 
both was new to us. We felt that this might also challenge 
other instructors and identified a need organize the delivery 
of prework and homework exercises upfront. We all noticed 
places where students struggled with technical language that 
could easily be clarified in revision. Independent of the 
materials review by InTeGrate team members, we deter¬ 
mined that fact sheets would have been more successful if 
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TABLE III: Summary of primary strengths and weaknesses of the pilot module from faculty postpilot reflection. 


Element 

Strengths 

Weaknesses 

Unit 1 

• Inclusion of foundational scientific practices 
(observation and interpretation) 

• Systems thinking introduced in homework 

• Take-home exercise on Earth systems vague and 
confusing 

Unit 2 

• Hands-on exploration of soil properties 

• Student designed investigations and predictions 

• Activity extended beyond 50 min 

• Students struggled to connect soil properties to land 
management practices 

Unit 3 

• Discussion promoted among all students 

• Preconceptions confronted students by 
(metacognition) 

• Students struggled with unit conversions, which 
extended instructional time 

Unit 4 

• Examination of authentic, local soil data 

• Students challenged with drawing to scale 

• Resources lacked level of detail necessary for student 
success 

Unit 5 

• Modeling of complex processes 

• Practice with systems thinking 

• Substantial student interactions or discussions 

• Technical level of background reading challenging 

• Systems thinking not emphasized 

• Low student engagement with prescriptive worksheets 

Module overall 

• Soil science addressed in the context of agricultural 
sustainability 

• Promotion of reformed teaching practices 

• Promotion of scaffolding 

• Alignment with InTeGrate guiding principles 

• Clarity and consistency of learning goals 

• Workflow tracking 

• Not personally relevant to all students 


we better related systems thinking activities to climate 
change and erosion response and if we better emphasized 
the use and interpretation of local data throughout the 
module. 

Classroom Observations and Focus Group-Identified 
Strengths and Weaknesses 

Classroom observation records and focus group results 
from the pilot of module Units 2 and 3 (Class 1) and Units 5 
and 6 (Class 2) were provided to the module authors. RTOP 
scores were reported for two of the four observation periods 
and were 68 and 59 for Units 2 and 3, respectively. These 
both are in the student-centered range of Budd et al. (2013). 
Observers reported that all four units promoted student- 
initiated discussions and questions. The observer for Class 1 
said that during the pilot of Unit 2, students generated a 
general definition and effectively synthesized information 
from previous classes. Similarly, the observer for Class 2 
noted that students relied on knowledge from earlier work 
to improve their knowledge or synthesis of new material. 
During the Class 1 pilot, the observer noted that students 
altered their experimental design to explore various proper¬ 
ties of soil permeability and porosity. This observer also said 
that students were involved in "discussion to make 
predictions, observations, or estimations." This included 
students describing how humans and climate can impact 
erosion. Observations from classes also highlight strengths 
across units, with observer feedback noting that "both units 
are designed to effective (sic) foster active student partici¬ 
pation in the transmission and discussion of material," "This 
activity fostered good group discussion," and the activity 
provided "opportunities for students to take charge of the 
discussion and to shift the focus of the lecture." 

Similarly in the focus group interviews conducted in 
both classes at the end of the observation period, students 
generally said that they were learning content and appre¬ 
ciated having some flexibility to pursue open-ended inquiry. 
Student focus group feedback is summarized in Table IV. A 


student also commented on the complexity of the topic of 
climate change, exhibiting higher-order thinking as that 
student sought clarity. Several organizational and delivery 
issues were also described by students in focus group 
interviews. A few students expressed confusion over 
assignment workflow and wanted to have greater clarifica¬ 
tion of expectations. Worksheets may have been part of the 
confusion because they varied in delivery timing (prework, 
in-class, and homework). Several students also felt that 
worksheets could be reduced or better organized. The 
observer also suggested that during module revision, 
expectations could be clarified by providing learning goals 
for each unit up front using a standard format for each unit. 
During the Class 1 focus group, several students commented 
that it had been awhile since they had taken math, and one 

TABLE IV: Summary of student perceptions of strengths and 
weaknesses of the module determined from student focus 
group feedback. 2,3 = focus group following teaching of Units 2 
and 3. 5,6 = focus group following teaching of Units 5 and 6. 


Strengths 

Interactivity/hands on (2,3; 5,6) 

Learned content (2,3; 5,6) 

Open-ended inquiry (5,6) 

Quantifying the complex process of erosion (5,6) 

Weaknesses 

Ambiguity of learning objectives (2,3; 5,6) 

Unclear about structure (2,3) 

Unprepared for math (2,3) 

Lack of personal relevance (2,3) 

Fast pace/overwhelmed (5,6) 

Prescriptive worksheets (5,6) 

Lack of concrete solutions (5,6) 
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FIGURE 1: Student choice of "global challenge" topics for interdisciplinary geoscience literacy exam essay question 
student responses. When compared with responses for across InTeGrate modules during their pilot, only students 
who participated in A Growing Concern identified soils as a central topic. 


stated that they were "stumbling blindly" with erosion rate 
conversions and calculations. The observer suggested 
providing students with additional practice time or a cheat 
sheet. Another student commented that they would be more 
interested in learning about their hometown than about the 
town their college was in. The observer suggested including 
additional societal connections to increase relevance. 

Postcourse Interdisciplinary Essay 

As a final assessment of knowledge and scientific 
thinking skills gained from InTeGrate modules, students 
across the project completed a postcourse interdisciplinary 
problem-solving essay. They were charged to "Identify and 
describe a global challenge that society will likely face in the 
next 50 years." Essay topics for the entire InTeGrate cohort 
are compared with those for students completing the A 
Growing Concern module in two of the pilot classes are 
presented in Fig. 1. Students completing our module were 
the only students who included soil sustainability in their 
response (n = 4). This was the second most frequent 
response. Our students most frequently discussed climate 
change (n = 5) and water (n = 5). Climate was also central to 
the A Growing Concern module. In addition to the topical 


alignment of student responses to their participation in our 
module, we highlight other areas of alignment. One student 
wrote: "A global challenge we will face in the next 50 years is 
soil sustainability. Soil is responsible for feeding the global 
population and the threats to soil sustainability make the 
outlook grim. Erosion and nutrient loss are a big factor in 
failing soil sustainability." These concepts came directly from 
A Growing Concern. Several students considered agricultural 
sustainability but moved away from content considered in 
the module. One noted that soil sustainability challenges 
"will effectively crash our global economy if the political 
rulers do not make funding for farmers more readily 
available and benefits for using organic farming measures." 
Another stated: "Our soil affects so many different aspects of 
life and life cycles, such as farming, carbon cycle, and wildlife 
management." Another student connected the relationship 
between the soil and the phosphorus cycle, writing, "Two 
components of the Earth System that interact with one 
another include soil and the phosphorus cycle. The way we 
treat our soil could prevent the phosphorus cycle from 
flowing, which can affect soils and animals involved in the 
cycle. These systems interact on a daily basis." 
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One student accurately wrote that a grand challenge "is 
the amount of land that is eroding at fast rates due to human 
interaction" but incorrectly speculated that "the largest of 
these interactions is grazing." However, this inference may 
be derived from incorrectly linking unit topics (e.g.. Unit 2 
involved building a model for soil compaction, and 
subsequent units explored the spatial extent of erosion and 
the factors driving erosion rates). The author of this response 
exhibited a complex systems understanding of the responses 
to grazing, noting that "grazing leads to compacted soil, 
which can lead to flooding." Other answers showed similar 
understanding of systems interactions and topical intersec¬ 
tions (e.g., climate and agriculture) that are derived directly 
from material present within the module. 

DISCUSSION 

Module Success and Refinement 

Student summative fact sheets and interdisciplinary 
essay responses exhibit central elements of the rubric used 
to create pilot materials. Materials reviewers emphasized 
that the fact sheet assignment was largely aligned with 
central learning goals and InTeGrate guiding principles, and 
scoring supports this. In addition, all students passed the fact 
sheet assignment, and median scores of 80% or higher 
across courses suggest substantive mastery is possible in 
varied class settings. Furthermore, many student responses 
to the interdisciplinary essay were aligned with the focus of 
the module. Topically, more than one-third of the reporting 
students focused on soil or climate. Furthermore, many 
incorporated ideas directly from the module or connected 
knowledge attained in the module with other knowledge 
from their experiences. Even an essay response that 
incorrectly linked module ideas showed knowledge of 
systems thinking. This evidence suggests that the A Growing 
Concern module was successful at meeting intended learning 
outcomes. In addition, observations and focus group 
sessions highlight that the module successfully engaged 
students in student-centered soil exploration. 

Revisions made from pilot to publication on the 
InTeGrate website (Fortner et ah, 2015) addressed shortfalls 
observed in the module summative assignment and critiques 
made during observation periods and in focus group 
interviews. The following described revisions passed the 
final rubric review by InTeGrate Project members and an 
external soils content expert before publication. The impact 
of these revisions on student learning will eventually be 
considered as part of project-wide evaluation across 
additional implementation settings. 

Published units have a similar emphasis to those 
described in Table I, but some revisions were made to 
reduce weaknesses. While only a few edits were made to 
Unit 1 to address areas of student confusion in worksheet 
materials, we separated the last learning goal into two new 
learning goals to better align with separate activities. New 
learning goals focus on inferring how agricultural practices 
impact landscapes and soil sustainability and explaining the 
interaction of Earth's spheres within the context of 
agricultural systems. No changes were made to learning 
goals in Units 2 or 3, but the porosity and permeability 
activity in Unit 2 was revised to emphasize connections with 
land management. As part of the activity worksheet, 
students must now reflect on how compaction associated 


Soil Sustainability InTeGrate Module for Undergraduates 267 

with tillage might impact soil porosity and permeability. In 
addition, instructional tips were added to Unit 3 to 
encourage practice with rate calculations through The Math 
You Need When You Need It (Wenner and Baer, 2015). The 
last two learning goals were revised in Unit 4. They now 
state, "Explain how a chemical property of soil, percent soil 
organic matter, is distributed with depth" and "Compare 
local erosion rates, soil horizons, and percent soil organic 
matter with other sites and estimate differences in sustain¬ 
ability" (Fortner et al., 2015). Unit 4 activities and associated 
materials now emphasize comparing local soil conditions 
with other sites. We shifted revised materials away from 
having students examine several soil properties; instead, 
they now examine soil organic carbon in more detail. The 
greater local emphasis and comparison with other sites give 
students opportunities to examine sites of their choice. Local 
sites are included because research suggests that place-based 
investigations improve student understanding of the role of 
science in problem solving (Surpless et al., 2014). 

Learning goals for Units 5 remain the same, but 
activities and associated materials were edited to better set 
up the systems thinking skills that specifically link climate to 
soil response. In the revised unit, students diagram the 
impacts of climate change on soil response instead of 
spending the bulk of their time focusing on erosion 
variables. This revision directly addresses student confusion 
over technical language and prescriptive worksheets noted 
in the focus group interview. It also addresses the need to 
better connect climate conditions to soil response noticed by 
the fact sheet materials reviewers. Actively diagramming 
systems is associated with improved learning beyond 
narrative instruction alone (Wheat, 2010). The student who 
said they would rather think about their hometown than 
campus might benefit from allowing this option. 

While learning goals for Unit 6 remained the same, the 
final fact sheet rubric and associated chunking activity (i.e., 
students recall lessons from previous units needed in the fact 
sheet) were revised to better instruct students in areas of 
weaker performance. Original rubric categories were re¬ 
tained, but new elements better articulate the scope of 
authentic data that students were expected to use. New 
elements include describing the physiographic region and 
describing local soil properties. The climate-related rubric 
element is now separated into two categories, one that asks 
students to describe the impact of climate on precipitation 
and another that that asks them to predict how changes in 
precipitation impacts erosion rates in their region. To solidify 
both personal relevance and systems thinking, we added a 
reflection assignment after completion of the final fact sheet 
assignment. Students are asked to reflect on their knowledge 
of soil sustainability and identify aspects of the module that 
were challenging or interesting to them. They reflect on 
gained systems thinking skills: "How did I use systems 
thinking to understand impacts on soil sustainability?" 
Finally, minor edits related to providing workflow support 
and adopting standard learning goal language were made 
before publication in response to the student focus group 
concerns and our struggles during the pilot. Students are 
now able to track prework and homework expectations in a 
single document outlining all activities and the inclusion of 
student material pages. Tracking is now a standard feature of 
published InTeGrate modules. Clear expectations such as 
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these are associated with improved student learning 
outcomes (Kuh et al., 2011). 

Recommendations for Adaptation and Adoption 

Even though this study is limited to our pilot results, 
high-quality fact sheets suggest that adaptations of A 
Growing Concern and other InTeGrate materials should 
consider retaining alignment with the InTeGrate materials 
design and development rubric. The perspectives of profes¬ 
sionals solving sustainability challenges are complex, and 
curriculum needs to incorporate that complexity (McKeown, 
2011). Our module delves into the complexity of soil 
management throughout units by engaging students in 
different approaches to soil sustainability and by asking 
them to provide solutions in their final fact sheet assign¬ 
ment. This approach was used in a similar fashion by 
Balgopol et al. (2014); they engaged students in the 
complexity of agroecologic management by having students 
participating in a solution-oriented case designed to have 
each small team present evidence and perspectives from a 
single stakeholder and then work across teams to identify a 
consensus land management decision. A comparison of pre- 
and postresponses to an essay on land management 
decisions showed significant gains in their ability to describe 
systems and complexity (Balgopal et al., 2014). Using the 
interdisciplinary essay before a course begins, as well as 
after, might help instructors attain rapid insight into the 
learning gains of their students. 

Finally, we noticed the influence of the emphasis of our 
specific courses on how students engaged with the module. 
Water-related content was embedded in multiple learning 
activities included in all three pilot courses. Two courses 
included water-themed active learning units, and the third 
included an invited guest lecturer. Not surprisingly, our 
students emphasized water in their interdisciplinary essay 
responses. Some of the language from fact sheets and essays 
may relate to instructor emphasis. For example, Fortner 
described taking students on a field trip to a nearby organic 
farm, with this emphasis retained in some fact sheets and 
essay responses. Instructor stories that contain unique 
aspects of implementation are available as part of all 
InTeGrate modules. A link to these stories appears on the 
landing page for each module. We each intentionally 
designed our course to fit the module we implemented. 
Exploring fit may be useful to others considering imple¬ 
menting or adopting modules. 
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